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Abstract 

The Sato-Crutchfield equations are studied analytically and nu- 
merically. The Sato-Crutchfield formulation is corresponding to los- 
ing memory. Then Sato-Crutchfield formulation is applied for some 
different types of games including hawk-dove, prisoner's dilemma and 
the battle of the sexes games. The Sato-Crutchfield formulation is 
found not to affect the evolutionarily stable strategy of the ordinary 
games. But choosing a strategy becomes purely random independent 
on the previous experiences, initial conditions, and the rules of the 
game itself. Sato-Crutchfield formulation for the prisoner's dilemma 
game can be considered as a theoretical explanation for the existence 
of cooperation in a population of defectors. 

Keywords: Sato-Crutchfield formulation; evolutionarily stable strat- 
egy; Evolutionary games; Persistence. 
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1 Introduction 



Evolutionary game theory [1] is one of the important topics in Mathematics. 
Also it has many applications in Biology, Sociology and Economics. Gener- 
ally a game consists of a group of players (of which two players at each game) 
are interacting with each other obtaining payoffs. Every player is allowed to 
choose some strategies, according to some rules. There are many different 
types of games. Each of them has its own rules, strategies, properties and 
applications. 

Evolutionarily stable strategy (ESS) [1] is an important concept in popu- 
lation dynamics. Consider a population in which each individual adopts one 
of k possible strategies {P , j — l,2,...,k). A strategy /* is ESS if for all 
P 7^ /* (yet close to it), then 

(i) A(r,r) > A(P,r); or 

(ii) If A{r, P) = A{P, P), then A{P, P) > A{P , P), 

where A is the k x k payoff matrix. 

Recently, Sato and Crutchfield [2] have introduced a new formulation that 
is corresponding to mistakes or losing memory. 

Our aim is to study Sato-Crutchfield formulation both analytically and 
numerically. Stability and persistence of the solutions are studied. The per- 
sistence of a dynamical system [3] is defined as follows: 

Definition 1: A dynamical system is persistent if all cases where all Ui{0) > 
lead to 

lim infiij(i) > Vi = l,2,...,n. (1) 

t— >oo 

Also, we study the Sato-Crutchficld formulation for some different types 
of two-player games including hawk-dove (HD), prisoner's dilemma (PD) and 
the battle of the sexes games [1]. The Sato-Crutchfield formulation is shown 
not to affect the ESS of the ordinary games. But in some limit, choosing 
a strategy becomes random independent of the rules of the game and the 
initial conditions. 

The paper is organized as follow: In section 2, the Sato-Crutchfield for- 
mulation is explained. Both analytical and numerical results are added. The 
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Sato-Crutchfield equations is applied to HD game in section 3. Also the effect 
on ESS is studied. Section 4 is devoted for studying the Sato-Crutchfield for 
PD game and some of its modifications. In section 5, the Sato-Crutchfield 
equations are introduced to the battle of the sexes. Some conclusions are 
summarized in section 6. 



2 Sato-Crutchfield formulation 

Recently Sato and Crutchfield [2] have studied the dynamics of learning 
in multiagent systems, where the agents use reinforcement learning. They 
showed that, although the agents are not directly interacting with each other, 
a collective game arises between them through their interaction with envi- 
ronment. Such interactions can be modelled via replicator type equations. 

Consider two agents u and v with n possible actions, let Uj{t) {vj{t)) be 
a measure of the probability that the agent u{v) will take action j at time 
t. Sato and Crutchfield have shown that Uj{t),Vj{t) satisfy the following 
equations 



duj 
dt 

dvj 
dt 



Ui(5i [{Av)i - uAv] aiUi Ej Uj In , 
Vi(52 [{Bu)i - vBu\ a2Vi Vj In , 



(2) 



where A and B are payoff (reward) matrices for u and respectively, and 
«!, a2, Pi, (32 are nonnegative constants. If ai = a2 — 0, then one regains the 
standard asymmetric replicator equations [1,4]. 

We will comment on the case ai = a2 = a > 0. For simplicity only the 
symmetric case will be considered {A — B — diag (ai, 02), ui = vi, U2 = V2). 
Rescaling time to set Pi — 1, the system becomes 



(3) 



^ = ui {aiUi - aiul - a2M2) + auiU2 In 
^ = M2 (02^2 - aiuj - a2ul) + au2Ui In 
Restricting to the region > 0, i — 1,2, then 

Corollary 1: Bounded solutions of the system (3) are persistent. 

This means that even bad strategies will be adopted by some agents. To 

study this analytically assume 02 > > 0. In the case that a = 0, it is 
known that the best strategy is ti2 = 1, Mi = so for small a, we find 
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ui = exp ( ~— ) ) U2^l — ui 



(4) 



The solution (4) is asymptotically stable if 



a > exp 



(-^) (2ai + 4a2) 



(5) 



The persistence of all strategics can be understood intuitively by realizing 
that a > means memory decay [2]. This implies that some individuals 
may not know which is the best strategy hence they choose the bad strategy. 
Furthermore as a increases, Ui,U2 become close to each other, since more 
agents lose their memory. This agrees with the numerical simulations shown 
in Fig. (1). Both ui and U2 go to 0.5 for large a. At this point agents chose 
their strategies randomly without any regard to previous experiences. 



3 Sato-Crutchfield formulation for hawk-dove 



The hawk-dove game [1] is a two-player game, where each player is allowed 
to use the hawk strategy (H) or the dove strategy (D). It has many biological 
and economic applications. The payoff matrix is given as follows 



where g is the value of gain and c is the cost of fight. The max-min solution of 
von Neumann et al. [5] is to follow the D strategy for < g < c. If0<c<g, 
then the max-min solution is to follow the H policy. But this solution is not 
evolutionary stable, since a mutant adopting H strategy will gain much, this 
will encourage others to adopt the H policy. This will continue till the fraction 
of adopting the H pohcy (ui) becomes large enough to make the payoffs of 
both strategies equal, thus 



Applying the Sato-Crutchfield equations to HD game, the replicator equa- 
tion becomes 



game 




(7) 
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dui 



Ui 

2 



{g — c)uiU2 + gul + auiU2 In 



U2 
Ui 



(8) 



where M2 = 1 — mi is the D density. This case is numerically investigated for 
various values for g and c, see Fig. (2) for gr = 1 and c — A. The system is 
updated for 10^ time steps. The behavior of ui is studied versus a. As shown 
in Fig. (2), at a = 0, Ml = 1/4 in agreement with its ordinary value Eq. (7). 
As a increases, Ui increases rapidly and becomes close to 0.5. Therefore, 
for large a, agents choose their strategies (H or D) randomly with equal 
probabilities. This behavior is independent on the initial conditions. 

Sato-Crutchfield formulation does not affect the ESS. For example, con- 
sider the hawk(H)-dove(D)-retaliate(R) game, where the payoff matrix is 
given by 



9 

9 
2 
9 
2 



c> v>Q. 



(9) 



\{9-c) 


\{9-c) 

Let Ml (1x2) be the fraction of population adopting H (D) strategy, and M3 = 
1 — Ml — M2 is the R-density. Then Sato-Crutchfield equations for the HDR 
game become 



dMi 
"dT 



dM2 
"dl 



= Ml 



C 

■2 + 



Ml , U2{g + c) ulc 



-q;Mi 



,U2. 



M2 ln(— ) +M3ln( — ) 

Ml Ml . 



U2 
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Mile-- 



ufc 



— CM1M2 + a 



1 (^^\ ^ 1 ^^3^ 
Ml ln(— ) +U3\n{ — ) 

U2 U2 . 



(10) 



It is known that for a = 0, the ESS is R. This means the solution mi = M2 = 
is asymptotically stable. Also, for a > 0, the ESS is R, and the solution 
[ui = U2 = 0,M3 = 1) is asymptotically stable. Beginning from nonzero 
values for mi, M2 and M3, and for a > 0, the system always evolves to a state 
at which mi = M2 = M3 = 1/3. At this point, selecting a strategy becomes 
random with equal probabilities. But beginning from a zero value for the 
density of a strategy, it will not grow through the population. 
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4 Losing memory and the prisoner's dilemma 
game 

The existence of cooperation in a group of selfish individuals is an interest- 
ing problem in mathematics and social sciences. One of the models of this 
phenomenon is the Prisoner's dilemma (PD) game [1]. In this case the avail- 
able strategies are to cooperate (C) or to defect (D). The payoff matrix A is 
[{R, S), (T, P)], where T > R > P > S and 2R > T + S. For simplicity we 
take A = [(2, 0), (3, 1)]. 

The standard solution gives the unrealistic conclusion that players should 
not cooperate. However cooperation has been shown to be a solution if the 
game is modified into iterated PD, or PD on the lattice, or memory PD 
[6,7]. Recently another modification is given [8], where a third strategy is 
introduced so the allowed strategies are C, D and loner (L). In this case, the 
payoff matrix is given as 

1 a 



a 



(11) 



where 2 > 6 > 1 and 1 > a > 0. Wc call this game PDL game. 
The Sato-Crutchfield equations for the PD game are 



dui 

— — — „^ . ^ ^ „^ X—^I I . 

dr ^ ' \U\ 

where ui is the defectors density and U2 — 1 — uiis the cooperators density. 
It is known that for a = 0, the asymptotically stable strategy is (D) i.e. 
iti = 0, U2 — 1- This has caused the following question to arise: How can 
cooperation exist in a population of defectors (selfish population)? Sato- 
Crutchfield equations propose a new answer. Cooperation may arise due 
to memory loss which is equivalent to mistakes. But starting from a small 
number of cooperators in a defectors population what is the asymptotic ratio 
of cooperators? For very small positive a e.g. 0(0.01) it is straightforward 
to see that the equilibrium solution is Ui = exp(— l/a), U2 = 1 — Ui. This 
solution is asymptotically stable for very small a. But as a increases the 
solution approaches ui — U2 — 0.5 (as we have argued before). Thus in 
Sato-Crutchfield equations a small number of cooperators in a population of 
defectors will increase to reach 50% of the population for a not too large, as 
shown in Fig. (3). This is not the case for a = where it is known that 
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cooperation will eventually tend to zero. This shows the significant change 
between Sato-Crutchfield system and the corresponding replicator one. 
Applying Sato-Crutchfield equations to the PDL game, one gets 



1 dm _ 
Ml dt 




1 dU2 _ 
U2 dt 



(13) 



where Ui is the fraction of cooperators, U2 is the fraction of defectors and 
Us — 1 — Ui — U2 is the fraction of loners. For a = 0, it is direct to see that 
Ml = ^2 = is a neutral equilibrium for the system (13). For a > 0, all 
strategies coexist. Solving the system (13) numerically for a = 0.1, we get 
the fixed point (0.266, 0.155, 0.579). This confirms our conclusion that Sato- 
Crutchfield equations is an interesting approach to introduce cooperation in 
prisoner's dilemma type games. As a increases, the choice between the three 
strategies becomes random with equal probabilities. Therefore ui, U2 and us 
tend to 1/3. 

5 Sato-Crutchfield equations for the battle of 
the sexes 

Recently, the war between the sexes was emphasizes as being important 
[9]. The battle of the sexes [10] is an asymmetric game that simulates the 
confiict between males and females concerning their respective shares in their 
parental investment. The female has two strategies coy or willing (fast) while 
the male can be either faithful or philanderer. The male (female) payoff 
matrix A{B) is gives as 



Here we are applying the Sato-Crutchfield formulation to the battle of 
the sexes. The Sato-Crutchfield equations for the battle of the sexes are 



A 



-10 
-2 



B 



5 
3 



du 

at 



= u{l - m)(-10 + 12v) + aiu{l - u) In (i^) , 
ft=v{l- v){5 - 8u) + a2v{l - v) In (i^) . 



(14) 
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There is a unique internal homogeneous equihbrium solution E = (5/8,5/6) 
which is stable; but not asymptotically stable. Therefore in the case ai = 
a2 = 0, one gets oscillations. For the case ai > 0, 0^2 > 0, we get 

Proposition (1): The equilibrium solution of the system (14) is m = 
5/8 — (02/8) ln5, V = 5/6 + (a;i/12) ln(5/3), and it is asymptotically sta- 
ble Vai > 0, 0:2 > 0. 

This shows that in this case there is a significant change between Sato- 
Crutchfield system and the corresponding replicator one. 

6 Conclusions 

Although the Sato-Crutchfield formulation does not affect the ESS for a 
game, there is a significant change between Sato-Crutchfield formulation and 
the other formulations of games. Sato-Crutchfield formulation is correspond- 
ing to losing memory. In some limit, choosing a strategy becomes purely 
random independent on the previous experiences, initial conditions, and the 
rules of the game itself. This behavior is observed in all the studied systems. 
Sato-Crutchfield formulation can be considered as a theoretical explanation 
for some aspects in game theory, like the existence of cooperation in a pop- 
ulation of defectors. 
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Figure 1: For Sato and Crutchfield equations, the densities Ui,U2 are plotted 
versus a. Both ui and U2 go to 0.5 for large a. At this point agents chose 
their strategies randomly. 
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Figure 2: In the Sato-Crutchfield formulation for the hawk-dove game, the 
density of hawks ui is plotted versus a. At a = 0, = 1/4 in agreement 
with its ordinary value. As a increases, ui increases rapidly and become 
close to 0.5. 
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Figure 3: Using the Sato-Crutchfield equations for the prisoner's dilemma 
game, a small number of cooperators in a population of defectors will increase 
to reach 50% of the population as a increases. 
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